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Report of the ITA Findings to the NESC 
1 Identification 


ITA #: NESC-RP-001 


Requestor Name: William F. Townsend 
NASA Goddard Space Flight Center (GSFC) 
Deputy Director 

Requestor Contact Info: (301)-286-5066 
william.f.townsend®nasa.gov 
Code 100 

Short Title: CALIPSO Spacecraft Proteus Propulsion System Assessment 

Description: Personnel hazards associated with Proteus Hydrazine propulsion bus once loaded 
(launch-36 days) 

Date Received: 1 0-22-03 

Date ITA/I Initiated: 1 1-06-03 

NESC Chief Engineer (NCE) Assigned: 
Michael Hagopian 

NCE Contact Info: 301-286-6732 

Lead Assigned: Dr. Richard J. Gilbrech 

Lead Contact Info: 757-864-3303 

Date IT A/I Concluded: 1-27-04 


2 Executive Summary 

The CALIPSO spacecraft is scheduled for launch on a Boeing Delta II rocket from Space 
Launch Complex-2 (SLC-2) at Vandenberg Air Force Base (VAFB) in 2005. CALIPSO 
uses an “off the shelf’ hydrazine-fueled Proteus propulsion bus manufactured by Alcatel 
Space Industries. The bus is provided by the Centre National d'Etudes Spatiales (CNES) 
as part of its in-kind contribution to the joint mission. While an identical bus was flown 
in 2001 on the Jason- 1 spacecraft, concerns have been raised by GSFC safety and 
engineering that the Proteus bus does not meet NASA fault tolerance design guidelines 1 
or all of the Air Force Eastern and Western Range (EWR) requirements 2 , thus posing an 
unacceptable hazard to processing personnel. The Air Force EWR, Kennedy Space 
Center (KSC) Expendable Launch Vehicle Office, and Langley Research Center (LaRC) 
are all in agreement that the spacecraft is safe to process and launch given the planned 
spacecraft integrity testing and operational controls in place. GSFC believes the risks 
from these potential events have been incorrectly classified and has recommended 
additional measures to mitigate personnel hazards assuming the undesired events will 
occur. 
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The scope of this effort was a review of the Proteus propulsion bus design and an 
assessment of the potential for personnel exposure to hydrazine propellant. Loss of 
mission, spacecraft or launch facilities is obviously an undesired outcome, but was 
purposely placed outside the scope of this assessment. The duration of this assessment 
was two months. Specifically reviewed were the potential for leakage from the five (5) 
mechanical fittings on the Proteus bus, potential leakage through the thruster valves and 
the potential for an inadvertent firing of the thrusters. These personnel hazards exist only 
during the period when the system is filled and pressurized until launch (approximately 
36 days). Material from a variety of sources was reviewed and a site visit was made to 
VAFB to review the payload processing facilities and Delta II pad where CALIPSO will 
be processed and launched. It should be noted that key CNES information requested for 
this assessment through the GSFC program office was not provided (ref. Appendix A). 
This fact limited the review team’s ability to draw conclusions based on objective 
evidence and formed the basis for many of the requirements. 

The NESC acknowledges that welded joints are superior to mechanical fittings in 
preventing leakage but attention to workmanship and proper verification of the joint 
integrity is required for both. Mechanical fittings do afford a greater degree of flexibility 
in the assembly and repair of tubing systems. However, a thorough risk assessment must 
be conducted early in the design process to arrive at a configuration that presents the 
overall minimum risk to personnel, the mission and the environment. During the course 
of the review it was noted that the hydrazine system does not have a tank isolation valve. 
The NESC team acknowledges that the omission of a tank isolation valve in the 
propulsion feed system is less safe during ground operations than a system that has the 
capability to isolate leaks; but while one may be safer, both can be made safe through 
proper hardware development and launch site processes. Again, a thorough risk 
assessment must be performed when designing the spacecraft to make these configuration 
decisions. 

At this time, the NESC cannot objectively conclude that the Proteus bus as designed 
poses either acceptable or unacceptable risk to personnel. The program must adequately 
address all eleven (11) requirements stated in this report before the NESC can conclude 
personnel risk is acceptable. The requirements call for program review and approval of 
CNES assembly and acceptance test procedures and verification that the planned 
acceptance testing and integrity checks are performed by CNES before hydrazine is 
loaded into the system. Further, verification of the planned operational controls (e.g., 
leak detection, alarms, installation of the thruster arm plugs, personnel controls and 
minimizing spacecraft operations once loaded) are required to mitigate the risks to an 
acceptable level. Compatibility of hydrazine with the Voi-Shan nickel conical seals will 
be determined through an ongoing series of tests being conducted by Aerojet and test 
results will be documented in an addendum to this report. 
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3 Detailed Description of the Problem 


CALIPSO is a joint science mission between the CNES, LaRC and GSFC. It was 
selected as an Earth System Science Pathfinder satellite mission in December 1998 to 
address the role of clouds and aerosols in the Earth's radiation budget. The spacecraft 
includes a NASA light detecting and ranging (LIDAR) instrument, a NASA wide-field 
camera and a CNES imaging infrared radiometer. 

The issues addressed in this assessment involve the Proteus spacecraft bus provided to 
CNES via subcontract with Alcatel Space Industries. This bus is identical to that flown 
on the Jason-1 mission launched in December 2001 on a Delta II from VAFB. NASA’s 
Jet Propulsion Lab managed the Jason- 1 mission. Issues on CALIPSO are associated 
with the Proteus hydrazine propulsion system used for orbit corrections depicted in 
Figure 1. The system has five (5) mechanical MS-33656 37° Army/Navy (A/N) fittings, 
one located at each of the four (4) 0.225 pound-force thrusters (Astrium model CHT IN) 
and one at the outlet of the ten (10) gallon hydrazine tank manufactured by Rafael. All 
other connections in the hydrazine system are welded. 



j 

Figure 1. Schematic of CALIPSO Propulsion System 
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Three key issues have been highlighted: (1) use of mechanical fittings instead of welded 
joints for propulsion system fluid connections, (2) the potential for hydrazine leakage 
through thrusters and (3) the potential for inadvertent thruster firing. Personnel risks 
associated with these issues are: 

• Toxic exposure to hydrazine leakage from the mechanical fittings 

• Toxic exposure to un-reacted hydrazine in the thruster exhaust via leakage 
through the thruster valves or inadvertent thruster firing 

• Fire potential from hydrazine leakage and subsequent contact with incompatible 
spacecraft materials 

• Fire potential from thruster hot gas exhaust igniting combustible spacecraft 
materials 

4 Causal Factors 


NESC focused on the three key issues as stated above. A detailed assessment of the 
causal factors that could potentially lead to a catastrophic event can be found in the 
NESC-developed fault tree (ref. Appendix B). A more general discussion follows. 


Leakage through the mechanical fitting can be influenced by a number of design, 
environmental, assembly and processing factors. The design of the fitting must provide a 
consistent clamping force sufficient to provide sealing integrity in the environment to 
which it will be exposed. Key design factors include adequacy of structural/mechanical 
design margins and compatibility of material selections of the various A/N fitting 
components. Environmental factors that could influence leakage include temperature, 
pressure, vibration and shock. The environmental factors must consider the flight 
mission as well as those induced during spacecraft transportation and during ground 
processing. Assembly and processing factors that must be considered include proper 
torque application, potential for the introduction of contamination in the assembly and 
potential damage induced during assembly. A comprehensive qualification and 
acceptance test program can both certify the design for these conditions and verify the 
adequacy of the assembly process. 

Leakage through the thruster can also be influenced by a number of design, 
environmental, assembly and processing factors. Flow control valves located upstream 
of each thruster physically control propellant flow to the thruster catalyst bed. Key 
design factors for the valve include adequacy of structural/mechanical design margins 
and compatibility of material selected for the valve components. A number of 
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environmental factors can influence the performance of the valve and its propensity for 
leakage. They include temperature, pressure, vibration and shock, and must be considered 
for both the flight mission as well as those induced during spacecraft transportation and 
ground processing. Risk of leakage through the flow control valves can be significantly 
reduced with a comprehensive qualification and acceptance test program by certifying the 
design and verifying the adequacy of the assembly process. 

An inadvertent thruster firing could be initiated by unintentionally applying power to the 
actuation circuit, the drivers or the valve solenoids. The power source could be from the 
Ground Support Equipment (GSE) or an internal short in the spacecraft electronics. One 
additional influencing factor could be an inadvertent ON command by the spacecraft or 
GSE software. Typical safeguards used to minimize the potential for inadvertent thruster 
firing includes redundancy in the design which would require multiple failures to apply 
power and designs having multiple inhibits to prevent inadvertent application of power. 

5 NESC Risk Assessment 

5.1 Overview 

Anhydrous hydrazine (N 2 H 4 ) is a colorless, oily, flammable liquid that is miscible with 
water. It has a penetrating odor resembling that of ammonia with an odor threshold of 
3.7 parts per million (ppm). The National Institute of Occupational Safety and Health's 
immediately dangerous to life or health (NIOSH IDLH) limit is set at 50 ppm 3 . This is 
the recommended exposure limit to ensure that a worker can escape from an exposure 
condition that is likely to cause death or immediate or delayed permanent adverse health 
effects or prevent escape from the environment. The Occupational Safety and Health 
Administration permissible exposure limit (OSHA PEL) for hydrazine is 1 ppm 4 . This is 
expressed as a time-weighted average and is the concentration of a substance to which 
most workers can be exposed without adverse effect averaged over a normal 8 -hour 
workday or a 40-hour workweek. The American Conference of Governmental and 
Industrial Hygienists' threshold limit value (ACGIH TLV) is 0.01 ppm 5 and is expressed 
as a time-weighted average; the concentration of a substance to which most workers can 
be exposed without adverse effects. It should be noted that OSHA numbers are 
regulatory, whereas NIOSH and ACGIH numbers are advisory. NASA and the Air Force 
use the more stringent time-weighted TLV of 0.01 ppm as the limit for worker exposure 6 . 

Hydrazine liquid is extremely reactive and contact with incompatible materials can spur 
spontaneous combustion resulting in a fire. The explosive range of hydrazine in air is 
between 4.7 and 99 percent. Although hydrazine is detonable above concentrations of 
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4.7 percent in air, its low vapor pressure of 0.27 pounds per square inch absolute makes it 
more difficult to build up sufficient concentrations in a well-ventilated area 7 . 

The fact a hazardous event is unlikely to occur does not mean it cannot occur. For the 
three fault tree events considered in Appendix B (leakage of the mechanical fittings, 
leakage through the thruster valves and inadvertent firing of the thruster) a wide range of 
probabilities were derived by the GSFC and LaRC safety offices along with differing 
opinions on severity. There is subjectivity in determining an event probability as 
evidenced by the wide spread between the two safety offices. It was not feasible for the 
NESC to better quantify the probabilities through specific testing or analysis in the 
timeframe given. Hydrazine is a hazardous commodity and in the NESC assessment 
team's judgment, the possibility of leakage does exist and the event severity is 
catastrophic to personnel. Given this premise, the focus of this assessment was to 
minimize the probability that the current design could initiate these undesired events and 
ensure operational controls are in place to maximize personnel safety. 

5.2 Fault Tree Analysis and Mitigation 

The CALIPSO fault tree (Appendix B) and mitigation table (Appendix C) were 
developed to identify all possible initiators leading to the three events and provide 
mitigation rationale for these events. The methods of verification specified by NASA 
system safety standards are inspection, test, analysis, demonstration and similarity. 
However, for this assessment, demonstration (“We flew it before”) and similarity (“It 
worked on Jason- 1”) were not used as a means of closing fault tree events. Specifically, 
closeout of fault tree events could not be made due to the lack of availability of assembly 
level procedures and specifications. Events that could not be closed were incorporated 
into the NESC requirements. 

6 Overview of the Initial ITA Plan 


NESC reviewed the Proteus propulsion system design to assess the potential for 
personnel exposure to hydrazine from mechanical fittings or thrusters as well as the 
potential for inadvertent thruster firing. This assessment focused only on hazards present 
from the time the propulsion system is filled with hydrazine and pressurized to final 
closeout for launch, a period of about 36 days. Suitability of the system for flight and the 
potential for damage to flight hardware or launch facilities during ground processing 
were considered program risks and were not addressed. Likewise, this assessment did 
not address workmanship issues. It was assumed that stamp warranties, training, and 
process controls were properly implemented, hardware was built to print and work tasks 
were complete as documented. 
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Fault trees for each of the potential failures under assessment were developed as 
presented in Appendix B. Credible failure modes were identified and the controls the 
CALIPSO program has placed on those failures assessed. For failures the program has 
not already assessed or for which controls were deemed inadequate, independent testing 
was conducted to validate the program’s approach or additional controls were 
recommended. 

7 Modifications to the ITA Plan 

While decisions to incorporate or eliminate certain tests were made as the assessment 
matured, the basic ITA approach outlined above remained unchanged. Initially, NESC 
planned to build a flight fidelity mockup of the hydrazine tank, tubing and thruster setup 
to perform leak and vibration testing. After NESC requests for accurate configuration 
drawings were denied to the program by CNES, the value of the vibration testing was 
deemed questionable and dropped. A separate issue arose when conflicting data on the 
compatibility of hydrazine with the nickel seal in the A/N fitting was discovered. 
Compatibility tests, consultation with material compatibility experts and a literature 
search were added. Information from NASA’s White Sands Test Facility (WSTF) 
surfaced during final report preparation that resulted in the addition of a 36 day room 
temperature nickel seal soak test with results to be supplied as an addendum (see Section 
9.1.2 for details). 
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9 ITA Identified Alternative Courses of Action 

9.1.0 - Mechanical Fitting Leakage 

Properly welded fluid connections are inherently more reliable than mechanical fittings 
and should be incorporated in fluid propulsion system designs when possible. There are 
some circumstances, however, under which mechanical fittings offer an appropriate 
design solution. Ready interface to off-the-shelf parts, ease of maintenance, or potential 
for damage to soft goods during welding all may dictate use of threaded joints. MS- 
33656 type 37° A/N-fittings have been employed successfully in aerospace applications 
for many years and are acceptable for limited use providing they are (1) properly 
assembled, (2) validated by leak check as an assembly before use, (3) exposed only to 
temperature, pressure, vibration and shock environments for which they are certified, and 
(4) incorporate a secondary locking feature. The Proteus bus uses five such fittings; one 
at the hydrazine tank outlet and one at each of the thruster inlets (see Figures 2 and 3 for 
details). While lock-wire is used as a secondary locking feature, it is suitable only for 
preventing significant rotation of the B-nut and full disengagement of the fitting. Lock- 
wire alone will not prevent loss of joint preload 8 with subsequent reduction of clamping 
force at the sealing surfaces, and thus cannot be counted upon to prevent a fitting from 
leaking. 



Wire Locking 


Figure 2. MS 33656 A/N fitting installation detail 





B! 

NASL/ 


Document #: 

RP-04-01/ 
03-001 -E 

Version: 

0.01 

Title: 

CALIPSO Independent Technical Assessment/Inspection 

(IT A/I) Report 

Page#: 

11 of 29 




Figure 3. Exploded view of MS-33656-4 Fitting 

While the NESC was not provided specific qualification and acceptance test data for the 
CALIPSO Proteus bus, the NESC reviewed relevant test data from other propulsion 
system and component tests. In general these tests addressed qualification, acceptance 
and sensitivity of the MS-33656 type 37° A/N fittings for exposure to the environmental 
conditions of temperature, pressure, vibration, shock and assembly cycles. The following 
sections of this report summarize three test series conducted on the MS-33656 threaded 
fitting and the mitigating actions required to assure integrity of the CALIPSO Proteus bus 
fittings. 

9.1.0.0 - Review Voi-Shan Results of Evaluation Tests Conducted on Voi-Shan 
Conical Seals 9 

The objective of this test program was to demonstrate that the Voi-Shan conical seal 
would consistently seal a flared A/N fitting tube connection under varying applications. 
The test conditions were established in order to simulate very stringent requirements that 
could be encountered in actual usage. The environmental exposure conditions used in 
this test series are similar to the requirements for the CALIPSO spacecraft and in many 
cases bound them. 
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The test series used various sizes of the A/N 815 (fitting end, superceded by MS 33656, 
currently AS 4395), A/N 818 and A/N 819 (sleeve, superceded by MS 20819, currently 
AS5176), fittings and conical seals made in accordance with Voi-Shan standard VSF 
1015 manufactured from aluminum, copper, tin and nickel. Test conditions included: 

1 . Pressure at room temperature of 1 500-4500 pounds per square inch gage (psig) 
induced with helium, air and nitrogen and 6000 psig induced with hydraulic fluid 

2. Pressure at elevated temperature: 500 °F at 3000 psig-air 

3. Pressure testing during repeated disassembly/assembly: 1500 psig for 20 cycles 
and 300 psig for 30 cycles 

4. Sine sweep vibration testing with 3000 psig pressure 

5. Torque relaxation combined with time (6 to 360 hours), pressure cycling and 
vibration 

6. Shock testing: 20g's shock at 3000 psig helium, and 100 g’s shock at 3000 psig 
water 

7. Thermal Shock at 200 °F and 1500 psig-helium 

8. Pressure Impulse testing from 0-4500 psig at 35 cycles per minute for 100 cycles 

Several measurement techniques were used to measure leak rate depending on the tests 
being conducted. They included submersion in water or benzene, using a helium 
sensitive mass spectrometer, a visual inspection if liquids were being used as the pressure 
medium and pressure decay over time. Torque relaxation was measured by applying 
torque in the tightening direction and measuring the angle required to achieve to the 
original torque value. 

The published results show a robust design for all of the configurations tested within the 
conditions specified. Test results indicated that all of the joints remained sealed with no 
leakage measured. The torque relaxation tests did show some relaxation over time and 
after exposure to pressure cycles. In the pressure cycle testing the largest change in 
torque was 27% and this occurred after the first pressure cycle. Torque relaxation reduced 
to no relaxation after the third cycle and only showed 13% relaxation after the second 
cycle worst case. Results of vibration tests showed no torque loss after exposure to 
vibration. The assembly, checkout and acceptance testing processes conducted on the 
CALIPSO Proteus bus can mitigate the two conditions (time and exposure to pressure) 
that did show some torque loss sensitivity. 
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9. 1.0. 1 - Review of European Retrievable Carrier (EURECA) Spacecraft 
Qualification Test Report for the ENN 51200 - Size 4 Joint for High Pressure 
Application 10 

The objective of this test program was to qualify the design of ENN 51200 E joint (MS- 
33656 flared tube connection) for the use in EURECA program for high-pressure 
applications. Qualification environments that the high-pressure joint was required to 
withstand include loads induced from the vibration environment, thermal environment, 
operational pressure and pressure cycling, mounting (assembly torque) activities, proof 
pressure and burst pressure. The environmental exposure conditions used in this test 
series are similar to the requirements for the CALIPSO spacecraft. Three configurations 
of tubing length combined with the MS-33656 fittings were included in the test series to 
represent different load influencing factors. These include an angle length configuration 
to induce torsion and bending on the fitting, a torsion lever configuration to induce 
torsion on the fitting, and a straight tube length to induce axial loads on the fitting during 
the thermal testing. Leak checks were performed pre and post exposure to the loading 
conditions. The setup for leak testing included a vacuum test chamber, the test article, a 
helium leak detector, a vacuum pump and a helium pressure supply. The external leak 
rate criteria indicated failure if it exceeded lx KT 6 standard cubic centimeters per second 
(scc/sec). 

The published results of the test series indicated that there were no leak rate failures 
experienced for any of the three test configurations subjected to all of the loading 
conditions. The test report also emphasized that the loads induced by vibration in 
particular did not result in developing an external leak. 

9. 1.0. 2 - Review of Experiments on the Robustness of Separable Fittings 11 

The objective of this test program was to investigate the effect of off nominal or stressing 
conditions on various mechanical fittings to assess the likelihood of leakage. Stressing 
conditions used in the test series included vibration (30 g’s root mean square for 300 sec), 
thermal stress (exposure to cryogenic temperature), misalignment (2 degree offset), 
under-torque (50 % of nominal), and assembly in the presence of foreign debris (scoring 
of the sealing surface). The 'A inch size A/N fitting was one of four types being 
evaluated in the test series. Other types include a Dynatube fitting (beam seal tubing 
connector), a KC fitting (a modified A/N fitting with Teflon gasket), and a Swagelok 
fitting. Two test series were performed; one test series subjecting each fitting to various 
combinations of the stressing conditions and a second test series based on an eight row 
Taguchi matrix of conditions with the four fitting used in the first series plus one 
additional fitting called the GE fitting (A/N modified with a radiused or ball nose). 
Conditions for the second test series had also been modified based on results of the first 
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test showing insensitivity to some of the stressing conditions. The second test series has 
not been reported at this time so the following discussion is based on significant findings 
from the first test series. 

Preliminary results of the first series of tests showed a wide variability of the various 
fitting responses to off-nominal conditions and identified some insensitivities that are 
relevant to the CALIPSO assessment. Even though these tests cannot explicitly quantify 
the integrity of the % inch A/N fitting in the CALIPSO Proteus bus, data from these tests 
does show insensitivity or an inherent robustness of the A/N type fitting to some of the 
relevant causal factors associated with the hydrazine leak potential. It was determined 
that vibration and misalignment were not significant factors in the probability of leaks in 
the separable fittings as results showed negligible effect on the sealing qualities of the 
fittings. Surprisingly, the test series showed that vibration tended, if anything, to reduce 
leak rates more often than it increased them. In no case did a previously non-leaking 
fitting start to leak as a result of vibration and in 13 cases having the under-torque 
condition with a measurable leak rate, 1 0 had reduced leak rates after vibration. The two 
under-torqued A/N fittings with the largest pre-vibration leak rate had an increase in leak 
rate post vibration. With regard to misalignment, it was reported that the fittings appear to 
be sufficiently robust to withstand two degrees of misalignment prior to assembly. It was 
also reported that fittings that performed the most poorly were most sensitive to under- 
torque and contamination (scoring of the surface). Both the A/N and Swagelok fittings 
appeared to be sensitive to under-torque and surface scratches. However, appropriate 
inspection and assembly procedures and post-assembly acceptance testing can mitigate 
both of these sensitivities. 

9.1.0.3 - Summary of Historical Data Review 

MS-33656 threaded couplings show an inherent robustness if properly assembled, 
acceptance tested, leak checked and other appropriate checkouts are performed. Even 
though these test series do not constitute a qualification of these threaded fittings, they 
certainly demonstrate that the MS-33656 threaded coupling design provides adequate 
sealing integrity for the types of environments that the CALIPSO Proteus bus could be 
exposed to during its processing and flight mission. 

9.1.1 - CALIPSO Proteus Bus Fitting Assembly 

The torque level indicated by a gauge or wrench during fitting assembly does not 
represent actual clamping force at the sealing surface. In some cases, clamping force 
may not be sufficient to effectively seal a fitting, even though the B-nut is torqued to the 
specified level. Thread binding or physical interference with the wrench head can result 
in such a “false torque” condition. Mechanical fittings must be lubricated slightly to 
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prevent galling and minimize the possibility of false torque. Quantity and location of 
lubricant must be controlled to ensure not only that it is applied but also that it is applied 
only to moving parts and not to a sealing surface. Lube on a sealing surface may fill a 
scratch or other discrepancy allowing a fitting to pass leak check, only to be washed away 
or dissolved in the presence of liquid propellant creating a void that leads to a leak. 

CNES has indicated that lubricants were used in the assembly of the Proteus bus, but 
NESC was not provided copies of assembly procedures or specific data to indicate where, 
or in what quantities the lubricants were applied. 

As an overarching statement, any procedure review or procedure development done in 
response to the following eleven (1 1) NESC requirements should consider not only 
engineering content but also the clarity or “workability” of the procedure from a human 
factors perspective. That is, care should be taken to ensure the procedures clearly convey 
the author’s intent without ambiguity that could confuse the operator and lead to an 
unintended outcome. 

NESC-R-001 - Program shall demonstrate that Alcatel training and/or assembly 
documentation provided for proper lubrication of fluid fittings during assembly. 
Assembly procedures shall clearly delineate the type, quantity and location where 
lubricant was applied and ensure sealing surfaces were kept dry and free of any 
contaminant 

Fittings must be visually inspected before assembly to ensure no discrepant condition 
exists that might lead to leakage. Damaged threads, burrs or machining marks may cause 
galling and subsequent false torque. A contaminant on a sealing surface may not be 
detected during leak checks, but be washed away or dissolved in the presence of liquid 
propellant creating a void that leads to a leak. NESC was not provided copies of 
assembly procedures documenting Proteus bus pre-assembly inspections. 

NESC-R-002 - Program shall demonstrate that Alcatel training and/or assembly 
documentation provided for a visual inspection of fluid fittings prior to assembly. 
Assembly procedures shall ensure components had no visible defects and sealing 
surfaces were clean and dry. 

9.1.2 - Material Compatibility 

Fault tree assessment highlighted the potential for component failure as a result of 
material incompatibility. There was some conflict among the various sources consulted 
concerning the compatibility of nickel used in the MS-33656 fitting conical seals and 
hydrazine 12 ' 131415 . Materials experts at WSTF were consulted who indicated that 
decomposition of hydrazine when exposed to nickel is accelerated at temperatures above 
212 °F 16 but the small amount of surface area exposed in this application was 
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insignificant to make decomposition a concern. The possibility of corrosion exists in the 
long-term, but it should not lead to leakage resulting in personnel exposure in the 36-day 
period under assessment. The fact Voi-Shan seals are not plated is also favorable in this 
regard. However, since there were some lingering questions regarding compatibility and 
no evidence Alcatel conducted any definitive testing before incorporating nickel seals in 
the design, NESC elected to run a series of independent tests to ensure the seals and 
propellants were compatible. Aerojet was commissioned to conduct an accelerated aging 
test of the Voi-Shan nickel seals at elevated temperature and pressure, along with a room 
temperature "beaker soak test." The accelerated test will yield quick results, while the 
room temperature test will serve as a control to verify any positive evidence of 
decomposition is not due only to a temperature/pressure environment unlikely to be 
experienced by the spacecraft. Complete details of the Aerojet testing are included in 
Appendix E, and results will be provided in an addendum to this report. 

9.1.3 - Post-Assembly Leak Checks 

Leak checks provide confidence fluid fittings have been properly assembled and validate 
the overall integrity of the joints. They must be conducted at flight pressure, using media 
no more viscous than the propellants themselves and instrumentation suitable for 
detecting leaks at the smallest allowable level. Given the relatively low internal volume 
of the CAL1PSO spacecraft and Delta-II launch vehicle fairing, hydrazine leakage at a 
detectible level may result in an accumulation that violates the OSHA PEL of 1 ppm 
during the 36-day period between propellant servicing and launch. The industry-standard 
approach to such situations is to conduct leak checks at flight pressure with helium using 
a mass spectrometer as a detector. Helium leak checks provide significant margin 
(approximately three orders of magnitude) over liquid hydrazine leakage. Therefore a 
system verified leak tight with helium (<10' 6 scc/sec) will be leak tight for hydrazine 
unless a sufficient upsetting event occurs to change the status of the fitting 7 . 

CNES has indicated helium leak checks of the Proteus bus were conducted on a fitting- 
by-fitting basis after initial assembly. Total system leakage will be measured with an 
encapsulated helium mass spec before integration of the propulsion bus and again after 
environmental testing of the spacecraft. Specified limit for these tests is 8.4x1 O' 5 
scc/sec 17 . A final 12-hour decay test will be performed at the launch site before 
propellant servicing. NESC was not provided any other details regarding the leak test 
methods, specifications (including derivation of the 8.4x1 O' 5 scc/sec limit), or detection 
equipment to be used for these tests. Bagging and long duration mass spectrometer 
measurements at both high and low pressure would provide maximum confidence that 
fittings do not have small but growing defects that could eventually leak hydrazine 7 . 
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NESC-R-003 - Program shall demonstrate that the Proteus bus mechanical fittings are 
rigorously tested using techniques adequate to validate system integrity. Leak check 
procedures shall specify test method, equipment to be used, media, test pressure and 
allowable leak rate. 

While CNES indicated spacecraft environmental tests would simulate qualification-level 
vibration and thermal loads, NESC was not provided specific data describing the test 
series. If the acceptance test loads envelope shipping, transport and handling loads 
expected from propellant servicing through launch, the post-environment test leak check 
will serve not only to certify the assembly for the expected flight environment, but also as 
an effective screen for any fitting that may have passed initial leak checks at low (false) 
torque. During the site visit, VAFB relayed that the highest shock loading recorded 
during transport of a spacecraft was 0.6 g’s. By comparison, the low frequency Delta II 
launch environment is 40 g’s with high frequency response up to 2,500 g’s 18 . Acceptance 
testing to these or higher levels would certainly envelope the expected ground processing 
loads. 

NESC-R-004 - Program shall demonstrate that thermal and vibration loads applied to 
the spacecraft during environmental tests envelope conditions it will experience from 
servicing through launch. 

9.1.4 - Handling Environment 

Fluid fittings could be loosened if subjected to significant internal pressure or thermal 
transients. The period of highest vulnerability is during dynamic testing, especially 
propellant servicing, when pressures are cycled and the potential for flow-induced 
vibration exists. There is no indication that CALIPSO Proteus bus fittings will be 
subjected to cyclic thermal or transient pressures significant enough to cause leakage, and 
the induced vibration potential is minimal given the short line lengths and low flow rates 
involved. However, since the CALIPSO servicing procedures were not available for 
review, NESC was unable to assess controls placed on temperature, pressure and flow 
transients during hydrazine loading. 

NESC-R-005 - Program shall demonstrate that servicing procedures adequately 
control temperature, pressure and flow rates to minimize the potential for leakage. 

Even with all controls in place, the possibility of leakage still exists. Consequently, the 
program must take all reasonable precautions to ensure the spacecraft is monitored and 
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personnel can be safely evacuated in the event of a leak. Industry-standard measures 
include a mix of fixed and portable vapor detectors capable of monitoring in the 
appropriate range, area-warning systems and fixed control areas limiting the number of 
personnel with access to the spacecraft. 

A site visit to VAFB was performed on December 17, 2003, to review the two potential 
payload processing facilities that will be used for CALIPSO and the Delta II launch pad 
“white room.” A map of VAFB locating the various facilities is included as 
Appendix D. While the Astrotech facility was toured, the Spaceport Systems 
International (SSI) facility was under a security lockdown and was inaccessible. 
Hydrazine detectors used in the Astrotech facility can resolve leaks down to 0.001 ppm 
and typically are calibrated and set to sense at 0.005 ppm or one half of the ACGIH TLV. 
The Astrotech fixed detectors are Zellweger Analytics SPM line powered units with 
0.005/0.010 ppm gas calibration keys while the portable units are SPM Z purge monitors 
with 0.005/0.010 ppm gas calibration keys. Both audible and visual alarms are tripped at 
0.005 ppm and the automated response system commands roof louvers open and air 
exhaust fans to maximum capacity. Portable detectors are used at the beginning of every 
work shift to sweep the area for leaks before personnel are allowed to enter. A drain 
trench completely encompasses the area where CALIPSO will be fueled and serviced, 
and can easily capture the 30 kilograms (approximately 8 gallons) of hydrazine in the 
Proteus system. Similar detection schemes with alarms are used at the pad white room 6 . 


The Astrotech payload processing facility fire protection system incorporates dry- and 
wet-pipe deluge systems designed to meet code requirements while protecting hardware 
from damage caused by inadvertent activation. 19 Facilities are equipped with UV and IR 
detectors for continuous monitoring of high-hazard areas as well as ceiling-mounted 
smoke/heat detectors. Hydrazine sensors have fire alarm set points at one quarter the 
lower explosive limit (i.e., Va x 4.7 or 1.175 percent hydrazine in air). These alarms 
communicate with the base emergency response units. If SSI is selected to process 
CALIPSO, the project should verily the SSI detectors and alarms meet or exceed the 
capabilities stated above for the Astrotech facility. 


Post-servicing operations in the vicinity of the CALIPSO spacecraft will be tightly 
controlled. “Amber light” operations will be in effect in the payload processing facility 
and at the SLC-2 launch pad white room. Per memo from the Air Force 30 th Space 
Wing 20 , “A flashing amber light indicates a hazardous operation is in progress in the 
controlled area. Non-essential personnel shall be cleared from the controlled area. 
Personnel shall not enter without permission from the safety official or in the absence of 
the safety official the entry control authority. Only mission essential personnel will be 
allowed near the spacecraft, all preventive measures will be instituted, facilities will be 









NASA Engme|^ 

Document #: 

RP-04-01/ 
03-001 -E 

Version: 

0.01 

Title: 

CALIPSO Independent Technical Assessment/Inspection 

(IT A/I) Report 

Page #: 

19 of 29 


verified acceptable to handle a maximum credible spill and emergency response will be 
available and on call.” 


In the judgment of the NESC assessment team, the mix of hydrazine vapor detectors, fire 
detection and suppression equipment and personnel controls are adequate for conducting 
safe operations in vicinity of the CALIPSO spacecraft. 

NESC-R-006 — Program shall verify that the controls at the processing facility and 
launch pad identified above are in place to monitor for leakage from the time 
hydrazine is loaded until final closeout for launch. Additionally, the program shall 
verify that spacecraft operations are minimized after hydrazine loading, and that 
provisions are made for area securing and the rapid evacuation of personnel should a 
leak develop. Further, the program shall coordinate with all other payioad/Deita II 
processing personnel to ensure the program’s approach for minimizing personnel 
exposure to potential hazards is properly integrated. 

9.2.0 - Thruster Leakage 

Thrusters selected for the Proteus bus are designed with normally closed series-redundant 
solenoid-actuated flow control valves manufactured by Moog. The thrusters are of a 
mature design. A schematic of the valve is depicted in Figure 4. 
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Figure 4. Moog Dual Seat Dual Servo Thruster Valve 

NESC concludes the potential for external leakage from the thrusters either internally 
(across the control valves) or externally (thruster casing or seal) poses acceptable risk to 
personnel providing the program conducts an adequate pre-servicing leak check of each 
valve. While the program did indicate such testing was planned, NESC was not provided 
a specific description of the test or its pass/fail criteria. 

NESC-R-007 - Program shall demonstrate that pre-servicing thruster leak checks will 
be adequate to validate system integrity. Leak check procedures shall test each valve 
independently and shall specify test method equipment to be used, media, test pressure 
and allowable leak rate. 

During a site visit to Aerojet Space Propulsion, an issue with Moog thruster valves 
similar or identical to the Proteus valves came to light. A manufacturing process change 
by Moog resulted in a recall investigation on suspect serial number valves 21 . The 
program was notified of this and was working to clear the CALIPSO Proteus bus valve 
set. 




H 


Document #: 

RP-04-01/ 

03-001-E 

Version: 

0.01 

Title: 

CALIPSO Independent Technical Assessment/Inspection 

(IT A/I) Report 

Page#: 

21 of 29 


NESC-R-008 Program shall verify’ that the Proteus Moog valves on CALIPSO do not 
have defective plunger assemblies. 

9.3.0 - Thruster Inadvertent Firing 

The Proteus thruster firing circuit incorporates a number of controls to ensure valves are 
not inadvertently opened causing a thruster to fire. NESC concurs the controls are 
adequate, but recommends further steps be taken to positively preclude the possibility of 
an inadvertent command during periods of dynamic testing, especially power-up. A 
schematic of the thruster wiring circuit is shown in Figure 5. It is worth noting that the 
Astrium specification sheet for the thruster lists nominal flow rate at 0.44 grams per 
second. Even with all four (4) thrusters firing at nominal flow rate, it would take 4.7 
hours to drain the 30 kilograms of hydrazine in the propellant tank. 


N2H4 



IN 

THRUSTER 


Figure 5. Thruster Circuit Schematic with New Test/ Arm Plugs 
(PM refers to spacecraft processor module) 
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NESC-R-009 - Program shall demonstrate that test procedures verify relays 16 and 1 7 
are open before power is applied to the spacecraft. Since the design incorporates 
latching relays, verification of the last stable state by data retrieval or written record is 
acceptable. 

NESC-R-010- Steps for inserting and removing test/arm plugs shall be explicitly 
called out in the ground processing timeline. Final installation for flight shall occur as 
late as possible; until that time, plugs shall only be installed as required for thruster 
valve testing. 

NESC-R-0I1 - Program shall verify that all thruster firing circuit inhibits function as 
designed. 

10 Conclusion 

It should again be noted that key CNES information requested for this assessment 
through the GSFC program office was not provided (ref. Appendix A). This fact limited 
the review team’s ability to draw conclusions based on objective evidence and formed the 
basis for many of the requirements. At this time, the NESC cannot objectively conclude 
that the Proteus bus as designed poses either acceptable or unacceptable risk to personnel. 
The program must adequately address all eleven (11) requirements stated in this report 
before the NESC can conclude personnel risk is acceptable. These requirements call for 
review of CNES assembly and acceptance test procedures and verification that the 
planned acceptance testing and integrity checks are performed by CNES before hydrazine 
is loaded into the system. Further, verification of the planned operational controls (e.g., 
leak detection, alarms, installation of thruster arm plugs, personnel controls and 
minimizing spacecraft operations once loaded) are required to mitigate the risks to an 
acceptable level. Compatibility of hydrazine with the Voi-Shan nickel conical seals will 
be determined through an ongoing series of tests being conducted by Aerojet and test 
results will be documented in an addendum to this report. 

The expected response from the CALIPSO program to the NESC will be an action plan 
indicating how the program will implement the eleven (11) NESC requirements using 
their in-line engineering, operation and safety organizations. NESC will approve the 
action plan and determine the adequacy of the program’s responses. As originator of the 
actions, NESC will provide status (open or closed) on each requirement at the appropriate 
CALIPSO milestone review prior to hydrazine loading. The program should use 
Appendix C as a guide to address the NESC’s requirements. 
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11 Minority Report 

The assessment team observed that there is no isolation valve downstream of the 
CALIPSO propellant tank. The GRC members were of the opinion that the program 
needed to address this issue in response to a specific NESC recommendation and offered 
the following: 

“The lack of an isolation valve in the Proteus bus design maximizes the 
potential for loss if any one of the three hazardous events were to occur, 
since there would then be no expedient means to stop the flow of 
hydrazine from the propellant tank. As a result, the worst-case failure 
effect is that most of the hydrazine in the propulsion system would be 
released, possibly causing a catastrophic event (personnel injury or fire). 

There is no evidence that a formal risk assessment was performed to 
address these three hazardous events related to the design decision to omit 
an isolation valve.” 

“ Minority Opinion Recommendation - Program should perform or make 
available a formal risk assessment to address the three hazardous events 
related to the design decision to omit an isolation valve. As part of 
including an isolation valve in the design, this assessment should 
consider the replacement of the mechanical fitting closest to the tank 
with a welded joint. ” 


Two NESC Review Board members concurred with including this recommendation in 
the final report. The remainder did not, however, so by Board consensus it was rejected. 
While a thorough risk assessment early in the design process might have led to a different 
design solution, an assessment performed today would not reduce the potential for 
leakage from the fittings or thrusters and thus would not help mitigate the risks associated 
with the current design. Instead of incorporating the suggested recommendation, the 
Board ensured the lessons learned from this study and documented in Section 12 
highlighted sound design solutions and underscored the need for thorough risk 
assessments early in the planning of any project. 


12 Lessons Learned 

Project managers should strive to ensure issues are surfaced and resolved, through 
independent assessment if necessary, early in the design process so technical changes can 
be effected with fewer cost and schedule implications. Thorough risk assessments must 
be performed to arrive at a configuration that presents the overall minimum risk to 
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personnel, the mission and the environment. Such assessments should be well 
documented, approved through a formal process, and made available for reference should 
questions arise as a project proceeds. 

When NASA is involved in missions with outside partners, the level of NASA insight 
and influence on non-NASA hardware design, verification and acceptance testing should 
be documented, clearly communicated, and carried as a project risk to be tracked. There 
was clearly confusion over certain safety requirements among the organizations involved 
in CALIPSO. The roles of various in-line and independent safety organizations should 
be clearly defined and their expectations documented as project requirements. Projects 
should then act to meet these requirements or, when warranted, process waivers with 
rigorous, documented, technical rationale. 

Properly welded fluid connections are inherently more reliable than mechanical fittings 
and should be incorporated in fluid propulsion designs employing hazardous 
commodities whenever possible. This requirement should be reflected in appropriate 
Agency-level design standards and variance accepted only when accompanied by 
appropriate risk trades and supporting technical rationale. 

Since lock wire does not prevent torque relaxation, it cannot be relied upon as a 
secondary locking device to prevent fluid fitting leakage. NASA or industry should 
spearhead development of a redundantly-sealed fluid fitting with an integral locking 
feature that, once engaged, will positively preclude loss of clamping force at the sealing 
surfaces. Ramped, inter-locking teeth between the inside rear of the B-nut and back of 
the tube end might serve this purpose if the ramp angle and teeth were sized to prevent 
nut rotation and loss of axial load with the fitting at full torque (ref Nord-Lock Bolt 
Securing System, Nord-Lock AB, Mattmar, Sweden, www.nord-lock.com.) 
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1 7-Dec-03 Julie Schneringer (KSC resident office at VAFB) provided shock 
information on transporter for previous missions and detailed 
ground processing timeline for Jason 1 . 
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17-Dec-03 Insufficient configuration data to make vibe/leak tests traceable 
to flight. Availability of Astrium thruster and facility to test not 
achievable in short term. Awaiting CNES data package. 


! 

I 


<0 

CO 

*5 

to 

“O 

o 

C 


CO 

□ 

E 

0) 

c 

o 

o 

< 

c 

a> 

E 

CO 

CO 

0> 

CO 

CO 

< 

o 

(0 

0. 

< 

O 

O 

(/) 

Ui 


< 

>< 

c 

0) 

a 

a 

< 


(0 

c 

o 

*5 

o 

< 

4-» 

c 

0) 

E 

CO 

0) 

CD 

CO 

CO 

< 

4-* 

o 

CD 

’o 

k. 

Q. 

O 

CO 

Q. 

15 

o 


(O 



(0 


co 




<6 n 


9 9 
> > 
o o 


i 

h* 

O 

CO 

UJ 

z 


*2* 

CL 

9 

<0 

Q. 

TO 

O 

o 

T3 

a> 

*2 




« 
c 
.2 
- CO 

a> S 

$ ° 
m 

f- *o 
.2 c/i 

o -9 

< O 


i 

s' 

CL 

O 

C 0 

Q- 

« 

O 


! 

CL 

8 

Q. 


S. I 


-o m 
<u a> 
S c 

i! 


c$ 


<U 

3 

T> 


g> 

'35 


c 

0 

1 


CD 

I 

P 

CO Q. 
UJ o 

z 2 

o f 

O Tj 

S § 

S o. 


<C o — o 

Z* J 5 » 

TO ^ « 

ill 

c ® 3 
<» E 

ssl 

S£i 8 

n 0) (A 

* 1 S 

s S. S 

05 "O CD 
C C 3 

i i| 

i 

« * « 
® <5 ® 
n jz 


sa lif 


9? 

o o 

35 




^r 

o 

i 

o 

CNJ 


CD 
-#— » 
CD 

■a 

CL 

3 

ro 

c 


Q 

UI 


CD 


D 

1 

2 

O 


o 


TO 

1 


£ CO 

9 9 

§ o 

3 3 

CM CM 


I 



^ S 8 

a a a: 


CO 

9 


<o 

CM 


U*> 


cS £ 

05 O p 




i 


D 

Ui 

CO 

o 


00 


O 


3 


i 

< 


Calipso Project Assessment Actions 


l 

I 

j 


_l 


"O 

C 

<D 

a. 

a 

< 



24-Nov-03 Provide safe life, stress, and fracture mechanics data for 14-Nov-03 • Action assigned - due Nov 24 Calipso Project 

propellant tank. In particular, since the tank presumably captures 
the elastomeric bladder in a hemispherical weld joint, the fracture 
mechanics analysis must Include an assessment of the residual 
stresses at this location. 
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Final Fault Trees for Independent Assessment of CALIPSO 
By: Ed Zampino and Bill Schoren at NASA Glenn Research Center 
January 8, 2004. 
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If the couplings possess structural defects such cracks, major internal flaws, or they are produced out of a material that was not specified in the design, 
this may result in external leakage. 

Excessive Temperature from some source may cause the fittings to expand and be under strain. This could cause fittings to crack (or fail) allowing 
leakage of N2H4. Although this condition is highly unlikely it has been included in the fault tree. 



Appendix B. CALIPSO Fault Tree Analysis 


O 


c n m 
00 00 
.5 ,S d 
c c o 

'in 

<L> O S3 

8 


LU 

> </> 

£ O 

Oh qj 

0> 

X) +* 

o 2 

If | 

!H 

-C > ' 

** O c 
oo a o 

•C S ta ■ 

5 .2 x 

Q r w 

^ s t: * 
W fi s 
c/5 n °" 

a u 9 


a> 

^ £ 

.t5 co 

S > 


<t> *5 - 

gs» 

>> 

a, S 3 

^£0 

‘3 <-2 g 

c3 

cr c i 

rn C +- 

2 

t ai ,h 


& 

a> 

& r o *5 

& .s 

-C 

o 

-o 3 o 

-4— > 

c cr c 

g e | 

s 


a § | 

-O o C3 
*rt <D ^ 

§fj 

<D tn rH 

_o on *£3 
^ 4J - 

*3 8 g 

^ ^ co 
«2 -22 <£! 

I'c § 

tin 

op ^ a 

£ >>£ 

a) 5 

is a» > 
^13 O 

fll • — < fcn 

| fe & 

0 £ « 

B. B.-° 

— s 

8. S 

w | 

oijt/2 5 

fo ® 

r> 

Cfl CO t- 

a c 2 

•o S *• 

1 s r 

s> i. >> 

op'ob'c 

.9 .g ° 
3 to _C 
Q S P 


§ -T3 <£ I 
fl 0) 

g .a ® 

c po a> 

S S3 -9 ' 

Big 

« -2 -g 

2 a*S 
w — £ 
1 2 S' 

.§>J:s' 

« ^ o 

■i -gST 

O g (O 

S 8 s 
o « *o ■ 

cfl O 

S CO Q 2 , 
x: ~ 
o ^ b 1 

* 2 ° 

fll c« 

•S c 8 1 
© «* £ 
2 £ , 

-8 12 I 
'111 
p-i* 

.2 8 T3 

, O O S 

$>2 i 

■o *o e 
'! § 8 
g 8 H 
& ^ « 
.5 co g 

c G <tt 
a> o © 


: £ © g 
! 2 g 8 

:"S1 2 
! 3 L 6p 
i > £ .5 

1 O to 

; (d a & 

; x> x ts 

? U 1 2 
© <3 ’S 

i O 'rC ^ 

! % & b 

i o ^ £ 

; e 

: .2 <*h g« 

j 43 ° 

! > a> S5 

: S 


£0-0 0 


I 


fail) allowing leakage of N2H4. Although this condition is highly unlikely it has been included in the fault tree. 

The thruster valves have to be designed to take the stress (forces) exerted from within by internal fuel line pressure (pressure of the N2H4). 

The thruster valves must be designed to withstand forces from launch vibration. There are other events that can expose the valves to shock such as 
equipment collision. 
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Appendix C - CALIPSO Fault Tree Mitigation Matrix 

Leakage of Mechanical Fittings 
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Appendix C - CALIPSO Fault Tree Mitigation Matrix 

Leakage of Mechanical Fittings 
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Appendix C - CALIPSO Fault Tree Mitigation Matrix 

Inadvertent Opening of Thruster Valves 
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electrically connected to the spacecraft power bus. 



Appendix C - CALIPSO Fault Tree Mitigation Matrix 

Leakage of Thruster Flow Valve 
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Place qty 16 nickel seals in hydrazine for parallel exposure test on nickel material only. 
Volume of hydrazine and seal quantity is outlined below. 
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